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(57)Abstract 

PROBLEM TO BE SOLVED: To reliably detect a marker 
by arranging a marker as a position index at the time of 
generating a composite reality space in such a 
positional relation where another player cannot observe 
a marker to be used when plural players observe the 
composite reality space in the case of arranging the 
marker in a real space. 

SOLUTION: A head mounting type picture display 
(HMD) 105 provided with a small video camera 103 and 
a head position posture sensor 101 1. etc., is provided in 
player processing parts 100 to 300 provided at each 
player respectively connected to a control part 400. 
Then a photographed video by the eyes of the player is 
superimposed with a virtual space picture and displayed 
on the HMD 105. In addition, a head position posture 
measuring part 101 detects a marker for position 
correction through the use of an inputted picture, and 
corrects a signal from the sensor 101 1 to estimate the 
posture, etc., of the player. In this case, the marker can 

> 

precisely be detected by arranging each marker in such a positional relation where another 
player cannot observe the marker to be used. 




http://www 1 9.ipdl.inpit.go.jp/PA 1 /cgi-bin/PA 1 DETAIL 



6/19/2009 



CLAIMS 



[Claim(s)] 

[Claim l]As a position index at the time of showing compound real space, it is a configuration 
method of a marker stationed to real space, A marker configuration method stationing a marker 
by physical relationship that a marker whom only other players should use is not observed when 
two or more players which observe said compound real space in mutually different movable 
regions observe said compound real space. 

[Claim 2]The marker configuration method according to claim 1, wherein a marker whom only a 
player besides the above should use is a marker with similar marker, color, shape, and size which 
the player concerned uses. 

[Claim 3]The marker configuration method according to claim 1 or 2 stationing a marker whom 
only a certain player uses from other players in a position which hides with a realistic object. 
[Claim 4]The marker configuration method according to claim 3, wherein said realistic object is 
arranged for application using said compound real space. 

[Claim 5]The marker configuration method according to any one of claims 1 to 4, wherein a 
marker to station contains a marker who owns jointly between two or more players. 
[Claim 6]The marker configuration method according to any one of claims 1 to 5, wherein said 
marker's color is common. 

[Claim 7] A mixed reality device performing calculation and/or amendment of position attitude 
information of a player using a marker stationed by the marker configuration method according 
to any one of claims 1 to 6. 

[Claim 8] A storage which stored the marker configuration method according to any one of 
claims 1 to 6 as a program which can perform computer paraphernalia. 

[Translation done.] 

DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to a marker configuration method with a marker's 
easy detection, and the mixed reality device using it, even when showing compound real space 
and two or more players share especially compound real space about the method of stationing the 
marker used as a position index. 
[0002] 

[Description of the Prior Art]In recent years, the research on the mixed reality (Mixed Reallity, 
MR) aiming at combination without the joint of real space and virtual space prospers. MR 
attracts attention conventionally as art which reinforces VR for the purpose of coexistence with 
the world of a virtual reality (VR) and real space which were separated from real space, and 
which were able to be experienced only in the situation. 

[0003]It is [ which realizes mixed reality ] a head mount display (HMD) which is typical as a 
device. That is, mixed reality is realized by compounding and displaying real space and virtual 
space on HMD.The optical see-through method which superimposes pictures, such as CG, on a 
transflective (see-through type) head mount display (HMD) as a method of MR usine HMD 



? e u C r°™ nin8 pictUreS ' Such 38 CG ' to ^ ima « e *"* P hotoed wi <h the video camera with 
which HMD was equipped, there is a video see-through method displayed on HMD 

[0004]New fields which are completely qualitatively different from old VR, such as a use of the 
health benefits shown to a medical practitioner as a use of MR as the situation of a patient's 
inside of the body is penetrated, and a use of the work assistance which displays the assembly 
procedure of a product on thing in piles at a factory, are expected. 
[0005]The art how to remove "shifting" between real space and virtual space is required in 
common from these application. "Shifting" shifted a position gap and time, it could classify into 
the qualitative gap, and many measures have been taken from the former about dissolution of the 
position gap which can be called most fundamental demand also in this. 

of image processing can apply comparatively easily as a method of 
amending a position gap especially in the case of the video see-through method MR the 
conventional proposal is made about the alignment using image processing 
[0007]The marker by whom seal attachment was specifically done by the color which is easy to 
detect by image processing to the prescribed position of real space is stationed, They are the 
method of calculating a view position by the marker position detected from the picture photoed 
with the camera with which the player equipped, the method of amending the output signal of a 
positional attitude sensor by the marker position in a picture etc * 
[0008] 

[Problem(s) to be Solved by the InventionJTo detect the marker in image data and presume the 

needs to appear at intervals of [ almost 
uniform ] a moderate size in a picture. Since the marker of sufficient number for calculation of a 
view position needs to be simultaneously detected in a picture, it is necessary to station a marker 
so that it may be observed at a to some extent narrow interval in a picture. 
[0009]On the other hand, in order to raise a marker's tracing accuracy and discrimination 
precision, it is necessary to station a marker so that it may be observed at a to some extent laree 
interval in a picture. 6 

[0010]If it is a case where a player is one person, it is not difficult but to station a marker so that 
such conditions may be fulfilled, and. In application with which two or more players share 
common compound real space, it may happen from a certain player that the marker stationed by 
regular intervals so that observation is possible is not observed by regular intervals from other 
players. 

two persons* player provides the marker of a color 
which is different for every player in the air hockey game which strikes the pack of imagination 
mutually on the table which is a realistic object, it constitutes so that a marker can observe in 
suitable arrangement and size from each player. 

[0012]However, if the player which shares the same compound real space increases the marker 
arrangement by classification by color will become difficult. Namely, by image processing in 
order for image processing to detect a color, although the color of a marker and a background 
object and the color of the marker between different users need to be colors which can be 
detected and separated easily, respectively, If the color to be used increases, it will become 
difficult to fulfill this condition and it will serve as a generation cause of an extraction error or 
the discernment error between markers. 

[0013] Drawing 10 is a figure explaining the cause of erroneous recognition at the time of usine 
the marker of a plural color. A horizontal axis shows red, a vertical axis shows green, a blue axis 
is omitted and drawing 10 is indicated, in order to simplify explanation. In the figure the field 



where the field A defines the marker's typeA (red marker) classification-by-color cloth, and the 
field B show the field which defines the marker's typeB (orange marker) classification-by-color 
cloth, respectively. Thus, when a player increased and used the similar color as a marker, in spite 
of having observed for example, the red marker with the position of a light source, or the angle 
of photography, it may be detected as a color (namely, orange marker) of the field B, and will be 
a basis of erroneous detection. 

[00 14] Very many markers will exist in real space by stationing two or more markers of a 
different color for every player, and the field of view of a player becomes complicated, and also 
when feeling compound real space, it becomes a cause which spoils presence etc. 
[0015]There is the purpose of this invention in the marker for every player providing a marker's 
configuration method in certainly detectable compound real space, even when solving an above- 
mentioned technical problem and sharing common compound real space by two or more players. 
[00 16] Another purpose of this invention has the number of markers in providing a marker's 
configuration method in the compound real space which does not increase simply, even if a 
player increases. 

[00 17] Another purpose of this invention is to provide the mixed reality device using the marker 

configuration method in the compound real space by this invention 

[0018] 

[Means for Solving the Problem]Namely, a gist of this invention as a position index at the time 
of generating compound real space, Two or more players which are the configuration methods of 
a marker stationed to real space, and observe compound real space in mutually different movable 
regions, When compound real space is observed, it consists in a marker configuration method 
stationing a marker by physical relationship that a marker whom only other players should use is 
not observed. 

[0019]Another gist of this invention consists in a mixed reality device performing calculation 
and/or amendment of position information on a player using a marker stationed by a marker 
configuration method of this invention. 

[0020] Another gist of this invention consists in a storage which stored a marker configuration 
method method in compound real space by this invention as a program which can perform 
computer paraphernalia. 
[0021] 

[Embodiment of the Invention]Hereafter, with reference to drawings, the desirable embodiment 
of the marker configuration method of this invention is described. Although this embodiment is a 
mixed reality game which three players perform by sharing the same virtual space, in addition to 
this, the marker configuration method in the compound real space by this invention is applicable 
to arbitrary uses. 

[0022]Drawing 1 is a figure showing the composition of the mixed reality device for carrying out 
the mixed reality game which applied the marker configuration method by this invention. 
Drawing 1 shows composition in case the number of players is three. The mixed reality device 
comprises the player treating parts 100-300 provided for every player, and the control section 
400 connected to each player treating part, and the player treating part connected to the control 
section 400 by the change in a player is also fluctuated. 

[0023]Input-and-output portions (102, 103, 105, 1011, 1021, 1022), such as a sensor with which 
the player treating parts 100-300 have the respectively same composition, and the body of the 
player was equipped, and a display, It is supplied from signal processing collected from the 
sensor, and this signal-processing result and control section, and also comprises a circuit part 



(101, 104, 106, 107) which generates the picture which should be carried out based on the 
information on a player, and should be displayed on a display. 

[0024]The composition of a player treating part is explained with reference to drawing 2 in 
which drawing 1, the kind of input/output devices with which each player equips in this 
embodiment, and a wearing place are shown hereafter. As shown in drawing 2 . each player 
equips with the head wearing type image display device (henceforth the following) 105 which is 
a display for compounding and displaying real space and virtual space on a head in this 
embodiment. Although a video see-through type or an optical see-through type may be sufficient 
as HMD in this invention, by the following explanation, the case where video see-through type 
HMD is used is made into an example, and it explains. 

[0025]The two small video cameras 103 are formed in the portion near the eye of HMD105. The 
image by the almost same viewpoint as the viewpoint of a player photoed with this video camera 
103 is supplied to the image synthesis section 106 later mentioned via the image input part 104, 
is superimposed with a virtual space picture, and is displayed on HMD105 of a player. The 
image input part 104 supplies an inputted image other than the image synthesis section 106 to the 
head position primary detecting element 101. 

[0026]The head of the player is equipped with the head position attitude sensor 101 1 which 
consists of magnetic sensors further, for example. It can equip with the head position attitude 
sensor 101 1, for example using HMD105. The output signal of the head position attitude sensor 
101 1 is inputted into the head position attitude meter side portion 101. The head position attitude 
meter side portion 101 detects the marker for position amendment using the picture supplied 
from the image input part 104, is amending the signal supplied from the head position attitude 
sensor 101 1, and presumes the view position of a player, and a posture. 
[0027]On the other hand, the arm of the player is equipped with the interactive operation input 
device 102. The interactive operation input device 102 is a device for performing command 
input, when it has the positional attitude sensor 1021 which detects the position of the part with 
which it was equipped, and a posture, and the switch (trigger) 1022 turned on and off according 
to a motion of mounting parts and a player performs predetermined operation. 
[0028]While two or more players (this embodiment three persons) avoid the attack from the 
enemy who shares the same compound real space and appears on compound real space in the 
following explanation, It is a game which competes for an enemy's number or mark which 
defeated the enemy, and was defeated by the time time or the damage by an enemy's attack 
reached the specified quantity, and the case where the following command input is possible is 
made into an example using the interactive operation input device 102, and it explains. 
[0029]- Command 1 (collimation command) 

The command which displays the line of aim which points to a collimation position on virtual 
space. In the state where the back of the hand was turned up, it inputs by locating a wrist up 
rather than an elbow. 

- Command 2 (shooting command) 

The command which shoots at the collimation position which a line of aim shows. It inputs by 
carrying out reciprocation moving of the arm (applying to a palm from an elbow) to order above 
fixed acceleration, after the line of aim has been displayed by the collimation command. 

- Command 3 (defense command) 

The command which defends the other party's attack. Where the back of a hand is turned to the 
other party, it inputs by turning a fingertip up. 

- Command 4 (reset command) 



The command inputted when inputting a collimation command again, after inputting a shooting 
command. An arm is inputted by changing into the state where it hung down downward. 
[0030]That is, as operation of the player in the game of this embodiment, the input of a 
command will be repeated in a cycle called a collimation command -> shooting command -> 
reset command -> collimation command, and a defense command will usually be inputted if 
needed in this cycle. 

[0031]The command inputted by the interactive operation input device 102 is supplied to the 
image generation part 107. 

[0032]The image generation part 107 transmits the head position of a player and attitude 
information which are supplied from the head position attitude meter side portion 101, and the 
command information supplied from the interactive operation input device 102 to the control 
section 400. Received from the head position and attitude information of a player and command 
information, and the control section 400, and also A head position, attitude information, and 
command information and model information of a player, The virtual space picture which should 
be displayed on HMD105 of a corresponding player is created using information, including the 
position of the obstacle arranged in the position of an enemy character, the move direction, state 
information, and space, shape, etc., and it outputs to the image synthesis section 106. 
[0033]The image synthesis section 106 combines the picture (real space picture) of the video 
camera 103 attached to the position near the viewpoint of a player, and the virtual space picture 
which the image generation part 107 created, and supplies it to HMD105 of a player. 
[0034]The control section 400 comprises the compound real space Management Department 1 
by which the above player treating parts 100-300 were connected, and the realistic object 
position measuring part 2. While distributing the information about the head position of the 
player which received from each player treating parts 100-300, and a posture, and the position of 
the interactive operation input device 102, a posture and command information, the compound 
real space Management Department 1, Game processings, such as a success decision etc. of 
generation of the enemy character displayed on virtual space, disappearance, control, and 
shooting by a shooting command, are performed. The model of an enemy character, a position, 
the move direction, and the information on states — whether it was defeated ~ are also 
distributed to all the users connected with the information on each player. 
[0035]In arranging the realistic objects 31-33 which become an obstacle of shooting in order to 
adjust the difficulty of a game as shown in drawing 3 . the compound real space Management 
Department 1 also manages the information about the shape and the position of these realistic 
objects 31-33. 

[0036]The perspective view of game fields (compound real space) which looked at drawing 3 (a) 
from a certain player, and drawing 3 (b) show the plan, respectively. In drawing 3 . the case 
where the three real space objects 31-33 have been arranged as an obstacle on the table (real 
space) 30 is shown. In this embodiment, as mentioned above, since the input operation of the 
collimation command using the interactive operation input device 102 is the operation which 
makes a wrist higher than an elbow, arrange the real space object on the table 30 which has the 
height about the height of the waist in a common form, but. It cannot be overemphasized that a 
table is unnecessary depending on the command input operation by an interactive operation input 
device. 

[0037]In drawing 3, the real space objects 31 and 32 are made movable [ immobilization and 33 
]. If there is a movable real space object, the difficulty of a game can be changed dynamically. 
For example, if the movable realistic object 33 moves to right and left at random speed, shooting 



of the enemy character which is a target will become still more difficult than the case where only 
the static realistic objects 31 and 32 are arranged. The compound real space Management 
Department 1 may perform the movement controls of a movable realistic object, or other control 
circuits may perform them. In this case, the compound real space Management Department 1 
will manage the model of a movable object, and that position will measure the object position 
sensor 331 formed in the movable object 33 by the realistic object position measuring part 2. 
[0038] A mixed reality device which was explained above is realizable according to the client = 
server system which uses the control section 400 as a server and to which it makes a client the 
player treating parts 100-300 (circuit part). By distributing and processing the processing about 
each player by a client, the change in a player can be coped with flexibly. If it states more 
concretely, a player treating part with the general purpose computer device which has a video 
input output function and a signal receiving function from a various sensor. The control section 
400 is also realizable with each player treating part, the interface which can be communicated, 
and the general purpose computer device which has a measurement signal receiving function 
from the object position measuring part 2. 

[0039]However, since it is necessary to perform the operation about three-dimensional image 
display in real time, it is preferred to use the comparatively high-speed computer paraphernalia 
which have the accelerator (what is called a 3D accelerator) etc. which specialized in such an 
operation. Communication with the control section 400 and the player treating parts 100-300 also 
has the preferred connection by a circuit with big capacity, such as 100BASE-T. If the capacity 
of a communication line is small, the fall of processing speed will become large with the increase 
in the number of players. 

[0040](Marker configuration method) Drawing 4 is a perspective view showing an example of 
the marker arrangement in this embodiment. In the game of this embodiment, although the 
obstacle by a realistic object is arranged, it becomes possible to fulfill the above-mentioned 
conditions, restricting the marker who enters in the moving range of each player by stationing a 
marker using the obstacle at a view. 

[0Q41]Drawing5(a) - (c) is the figure which extracted the marker who can see from the player 
A, B, and C in drawing 4, respectively, and was shown. Thus, although number observation is 
carried out, since [ being required for calculation of a respectively almost equivalent interval and 
a view position ] the marker for other players is not recognized visually, the marker seen from 
each player does not need to change a color. It also becomes possible to share the same marker 
among two or more players, as shown in drawing 4 . 

[0042]Drawing 6 (a) - (c) is a figure showing the marker observed from each player 
corresponding to drawing 5 (a) - (c), when not using an obstacle. The number of the markers 
whom each player observes decreasing dramatically with the marker configuration method by 
this invention, and coming to fulfill the above-mentioned conditions is lucidly understood from 
contrast with drawing 5 and drawing 6 . 

[0043]When a player increases, it can be coped with by the method of changing the shape 
(sectional shape, height, etc.) of a realistic object of providing a marker, or adding a color. Since 
one color is not necessarily assigned to one player like the former also when increasing a color, it 
becomes possible to station the marker to many players in few colors. 

[0044] Although it may depend on a help, it can opt for the determination of a marker's locating 
position by generating the model of a realistic object, and the model of the view position 
movable regions of each player beforehand, and searching for the range which is a range which 
is in sight from the target player and by which the look of other players is interrupted. It may be 



made to compute a position using the number and mapping rule of the marker who wants to 
provide. On the contrary, it may constitute so that obstacle shape and/or arrangement which 
fulfill such conditions may be computed. 

[0045]Even if it is a case where there is no object which can be used for arrangement of markers, 
such as an obstacle, A player becomes possible [ experiencing compound real space ] by 
arranging a realistic object in the position which does not become an application top problem, 
and hiding existence of a realistic object by a virtual space picture, without being conscious of 
the realistic object for markers. Concealment of the marker by a virtual space picture, etc. is 
explained in full detail later. 

[0046](A marker's detection) A marker's detecting method is explained below. Drawing 8 i s a 
flow chart which shows the flow of the processing which detects the marker (red marker) of 
typeA who has a color included to the field A shown in drawing 7 . The head position attitude 
meter side portion 101 performs each of marker detection processings and use of the information 
of the marker who detected. 

[0047]First, the picture photoed with the video camera 103 is acquired via the image input part 
104 (Step S701). And binarization processing is performed (Step S702). The pixel specifically 
contained to the field A shown in drawing 7 (a blue axis is omitted and indicated) is set to 1, and 
except [ its ] is set to 0. Namely, Ii: The i-th pixel Ri that constitutes input color image I, Gi, Bi: 
R which constitutes Ii, Value ITHi of G and B each color: The i-th pixel value RminA of a 
binary format image. GminA and BminA: R, G, and B which define the field A - each of 
minimums RmaxA, GmaxA, and BmaxA: R, G, and B which define the field A, if it is each 
maximum, For every Ii, 1 and ITHi corresponding to the other Ii are set to zero, and the binary 
picture ITH is formed in ITHi corresponding to Ii which fills RminA<Ri<RmaxA, 
GminA<Gi<GmaxA, and BminA<Bi<BmaxA. 

[0048]Subsequently, a labeling processing is performed to the binary picture ITH, and a marker 
block (cluster) is extracted (Step S703). And the center of gravity (Xn, Yn) and the area an of 
each cluster are computed (Step S704), and it outputs to the view position posture presumption 
module (not shown) of head position attitude meter side portion 101 inside (Step S705). In a 
view position posture presumption module, from a marker's absolute coordinate registered 
beforehand and the marker position (cluster center of gravity) detected from the picture, the 
output signal of the head position attitude sensor 101 1 is amended, and the view position and 
posture of a player are computed. 

[0049]In drawing 8, although the processing in the case of detecting the marker of one certain 
color (red) was explained, when the number of players increases and the marker of a plural color 
exists, the threshold at the time of generating a binary-ized picture is changed, and the detection 
processing for every color is repeated and is performed. 

[0050](A marker's elimination) As above-mentioned, if the marker's output of the head position 
attitude sensor 101 1 is originally exact, he is unnecessary. As for a marker's existence, in 
compound real space - there is a possibility of spoiling presence ~ when a player recognizes a 
marker, not being recognized is desirable. 

[0051]Therefore, in this invention, a marker is seemingly eliminated so that a player may not be 
made conscious of a marker's existence. As a method of eliminating a marker seemingly, 
although various methods can be considered, the load and the sense of incongruity of processing 
of a method which superimpose a virtual image on a marker and are displayed on a player are 
preferred from a small thing. 

[0052]Drawing 9 is a figure explaining a marker's erasing method. Drawing 9 (a) is a thing 



showing the marker to the player A stationed with the marker configuration method in this 
embodiment, and corresponds to drawing 5 (a). Thus, in order to eliminate the marker stationed 
at realistic objects, such as a table and an obstacle, before stationing a marker first, the place 
where arrangement is planned is photoed with video, a digital camera, a still camera, etc., and 
image data is acquired as a texture. And after arrangement of a marker, it replaces with the 
texture image which acquired beforehand the image data which is equivalent to a part for a 
marker part among the image data displayed on HMD of a player, or it superimposes, and image 
data is displayed (drawing 9 (b)). 

[0053]Thus, it becomes possible to concentrate on a game, without a player recognizing a 
marker's existence by replacing / superimposing image data. The image data prepared as a 
texture has it, when acquiring under the conditions (a light source position and a luminosity) 
actually used reduces the sense of incongruity on vision, but it is good also considering the 
texture image which eliminates the marker stationed at the realistic object of the same 
construction material as one common texture. [ preferred ] 

[0054]In the application which does not use an obstacle, like [ a case as the realistic object has 
been arranged only for marker arrangement, when it is difficult to replace / superimpose image 
data to each marker ] drawing 9 (c). The picture 91 which covers all realistic objects can also be 
used. In this case, what kind of image data is used should just set suitably with application. For 
example, what is necessary is to form a thing like the stage in virtual space, and just to use 
conversely, the data of the object photoed in real space in game application, using this picture 91, 
if it is the application for which you want to sense a difference with real space. Of course, it is 
also possible to consider it as the data of a three-dimensional object instead of two-dimensional 
image data. 
[0055] 

[Other embodiments] Although the information acquired from a marker was used in the above- 
mentioned embodiment for the error correction of the head position attitude sensor 1011, 
Without using the head position attitude sensor 101 1, it is also possible to search for the view 
position posture of a player only from the information acquired from a marker, and it is possible 
to apply this invention also to such a use. Although the above-mentioned embodiment was a 
mixed reality device of the video see-through method, it cannot be overemphasized that this 
invention can be applied also like the mixed reality device of an optical see-through method. 
[0056]In an above-mentioned embodiment, although the trigger 1022 was not used, but the 
motion of an interactive operation input device was detected and the command was inputted, a 
player may input various commands using the trigger 1022, or detection of operation and turning 
on and off of a trigger may be combined. A switch which is turned on and off according to 
operation of a player can be built into an interactive operation input device, and it can also use 
for it as a trigger. 

[0057]The purpose of this invention the storage (or recording medium) which recorded the 
program code of the software which realizes the function of an embodiment mentioned above, It 
cannot be overemphasized that it is attained, also when a system or a device is supplied and the 
computer (or CPU and MPU) of the system or a device reads and executes the program code 
stored in the storage. In this case, the function of an embodiment which the program code itself 
read from the storage mentioned above will be realized, and the storage which memorized that 
program code will constitute this invention. By executing the program code which the computer 
read, Based on directions of the program code the function of an embodiment mentioned above 
is not only realized, but, It cannot be overemphasized that it is contained also when the function 



of an embodiment which performed a part or all of processing that the operating system (OS) etc. 
which are working on a computer are actual, and was mentioned above by the processing is 
realized. 

[0058]After the program code read from the storage was written in the memory with which the 
function expansion unit connected to the expansion card inserted in the computer or the 
computer is equipped, It cannot be overemphasized that it is contained also when the function of 
an embodiment which performed a part or all of processing that CPU etc. with which the 
expansion card and function expansion unit are equipped are actual, based on directions of the 
program code, and was mentioned above by the processing is realized. 
[0059] 

[Effect of the Invention] As explained above, according to this invention, even if it is a case so 
that two or more players may share the same compound real space for the marker used in a 
mixed reality device for position ******, it becomes possible to arrange fulfilling required 
position numerical conditions. Since it is not necessary to change a marker's color for every 
player, even when the number of players increases, the probability of erroneous detection can be 
controlled. 

[0060]Since the real space object arranged from the first can be used, it also has the effect that it 
is not necessary to provide the object for stationing a marker depending on application. 



[Translation done.] 
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zbumb timm 3 ie»?r?-#EBm 

[ 6 ] w\U-?-tl<r>&mMX't> 6 .1 i: £f$8t 

t -r & m^a i mmm. 5 a v vfttMaeaa*?-* 

c mm. 7 1 it i jbMurass 6 *>v ^-rtuMzgrn 
w-tmsjjmizx^xm&ztitiv-ti&m^xr 

[000 1 3 
[0002] 

^*S-££BWi:L£«£«SI3i (Mixed Reallity, M 

r) icsf^affi&amicsro-o^. MRiisa?, « 
++wro?-< (vr) ^t#t3msrai:o*#$- 

gWfcU VRS:«3lt4a«i:LTaS$^TV^. 
[0003] «£SH!8£0!£-f SgBi: LTftfcW* 
*><9Ji. A7 Kv^yhT-fxrW (HMD)f* 



m (is-xjv-m 0v\vY~?*?yY¥4*'r\/<< (h 
MD) sm<nwi&*n$.-thjtfr i y-ziv-irg. 

b. HMD Lfc £?*;M L fcH«-f- 
[0004] MROffi&fc LT(i. &#<0fcrtO«tf* 

T%£to^xm^A&x*mt%foizntox$&?r 

httmm<r>Wm:b' . $£X<r)VRb\t$tmz£<% 

%^KWz%mimmtix^h. 

[0005] Ztie>V>miZtflX#MlZ$&tZtl&<r) 

-ftu m-ftiiz<m*im?k o , .r o*? 1 1 ^ t t 

[0006] mz h'r ^ i—X^-^M R , & 

TftiiffiEthimb ixm&tm<?&&tfimim 
[0007] ftftffjKii. ^n^ia<7)j5f«fiatc. am 

at, TU--\»3!i^L^^7T1i»Uv:iii«*>^«l 
SB Lfcv-afflRc J: o-CS^ffla^^-tS^?*, 

H^+^v-^fflatc .to Tffia«»-b ^^aj*m# 
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[^* { »^L«l:oi:-r-6illJffi] H«r-*+tf)-7-;* 
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[00113 -eo^ft, ^fBffl l-84 3 0 7#^« 
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#7-u-A»*^v-^^»a^ieafcA& zxwmx* 
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[0012] LfrLKtfb* n-y&msfoim-r 

v-^ilHHJjftcOfefcJrtffS&fta-- f 

too 1 3 3 @i ota, fflt&w-ttzm^tzisicom 
tsmmmiz^xmnthm-efii. miotz&^x. 

t y p e A (IfrfiV-tf ) OfeftftS-Jgij-f ftffl 
i*. ^JSBIiV-^ t y P e B ) Oft^ 

m<r)m*?m<r>fttiL\izx ~>xmz.if z& 

&tlXftlU$tiXl*o-5Hmm>). M&li&hi: 
tZiXlto. 

[oo 143 i^, Zrv-^ziuzntch&ni-ti 

zmmm ft z t <c x o x mmisnzimz 2 < <r> •? 
-Atftttthztiz**). Tis-^nvmtfjmiz* 

[00 1 5 3 *f&9!<9SWli. -B&9il!jH£«&U « 
STl— -We J: 0 *JI?)#te3ilS2|g!£ £3r*-ft#£T 

t . rv-^z k <?>v-fitffi&iz&iiiiiim%tiL&m 

zmz mz-?- *<D8.sxm z m-t 1 ztizhi. 

[0 0 1 6 3 *ffil<?>Wlcr)mi&. 7V— W*Jg 

an lt t v-tunrnwrnzmn ^-^i^rat; 

[ 0 0 1 7 3 ifc. *&^SO^BWtt*&HJJfc:J:ft« 

£^^£fcttftv-;&EB^£fi|fflL*:«£^ 
l8&B.£&m-hZklz}>&. 

[00183 

fsisrsv jmt 4 m>*im®mfix'W&-t ft «iR<or w 

tz&.&t&ztzmLt-i-hi-tEmiTmzft-f 
ft. 

[00 1 9j *7t, *fm^jM^siitt % *hbjov- 

^EB^£«i:oTEB3ftfc7-;*£fflvvCTU--\» 
tf>{iBtir$fi?)»aj&tf/a fcttJSiEfcfT oZt iftWib 

[ 0 0 2 0 3 *^(CJ; 5 

*^l«aait=45 ft ft 7-* EBmfrfc* 3 y t a - 



£#*ft. 
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-^EB^rft<o»itv^segS8fc:ov^TiKDWft. * 
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MMt&ZttfTZh. 
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r fc»»t ^tit^rv-^wm 1 0 o~ 3 0 0 1 . & 

7]s~>rmmizmmtit:iif\m&4 00 1 

[0023J 7V— wm*- i o 0-3 o ottweft 

R-vMfSLttLXto*)^ TU-^eomzgmztifc-t 

yvssmmw&znx&jim ( 1 o 2 . 103, 1 

0 5. 101 1, 1 0 21. 10 22) fc. -by^4> 

^StlftffiTU— V<o««5-7Et LTa^SBfc^-r 
^H«££jfrfft08S&# ( 1 0 1 . 104. 10 
6. 107) bfrt>m&2tii> . 
[ 0 0 2 4 3 WT. 0 1 t*Htt^S{ctj^T#7-W- 

vwmnmf&mmz. 02^ 

££IBJi<KS2IS!fc Sr^LT ft^oa^SB 
•Ci>ftHSil*Mil«^5IB (feiTHMDfcta ) 1 
0 5 £g»-f ft . *f|H^cfe v ^THMDIi t'-f * x-^ 

Wfft. 

[00253 HMD 1 0 5c7)B(ci5^|fS^(c(i, 2 OC0 
^t'r^^>7l0 3*^t^-CV^ft. d^t'r^ 
1 0 3-C«»$^. TP— V^)«^i:lJ<3r|SI- 

^«^»:«k ft»«Wi. ®«A^jffl 1 0 4 $rrt-LT8B£"f 
SSft^-JSail 0 6tc«IS$^i, (R152IBIH«i:Sg$ 
tlXTV— ^COHMD 1 0 5(C^$ixft. ifc. 9ft 
A2JS81 0 4JiH«^aJi 0 6£0flfetz. A^H«$-H 

ssttB^ajai 1 0 1 ^m-th . 
&&±ytfrt>Kcm&Q.Em$-tz>vi 0 1 1 

$*tTHft. HS5ffiB^-b^l 0 1 Hi, W^.tfH 

md 1 0 5 mmixmthz tm* h . mam. 
m±>*n 0 1 1 <^attfi#»in&ftB££fHiia5 1 
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A^ja 1 0 4 A^f&fc$ilftS«mvctiBffliBfJc7) 
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v-^&asu mmme^y^i o 1 1 x <om 

[0027]-^, rv-^<m\,z\t. m&mftxjjm 
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[0028] WT<oi»BB{cijv^Tii, «8<*>7V~f 

»£*ru mfsif>iwm<?)imizj;z>y*-i; 
mzmizm- & t tusk Lfci&os* s v ^ j^s* 

^^y-ATAo-C. ttfStlftA^B 1 0 2£EUT 
[0 0 2 9] -nvyh'i (spsn^^H) 

•WavfFfcAflLfctft. S«!i«S3vyH^A^-r 
^^TA^-fS. 
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^nv^Hli. UB&ftftl 0 7t;:»te$ft*. 
[0032] B«£j£l3t 1 0 7li. HgBttB«#?tf-a!ia5 

i o i *>^>ews?n47-v— froKa&fta&i^fM&fH 

kttiSliftA^&Bl 0 2j^&te?ft£3vyF^ 
£$SP8P4 0 0'MES£t£. rv-^amu^M. 

ahr. siart keb ? tuzmmnoM. . m&£<m 

WfcfflWC. MK+iris— \*OHMD 1 0 5(C^rf 



[0033] MfltefifcS 1 0 6 it. ru-^com&izft 
^QMizM^Hif^tLtit'f^iiMyl 0 305H« (H 
2IBJB«) k. ®«£«m0 7j^j£U:fe1g£|5JjI 
«i:££l£U TP-^HMD 1 0 5fc&tef 

[0034] $flWSf4 0 Ott, 3 £7* U~ 

aa? i o o - 3 o o ^««$iTJt a^sns(gnraaj 1 

k, $^MiBfti»«2fcj!^fcrt3ftTV^. 

Sl^SltPffsat l (J, 1 o o~3 o o 

b. ttiSSffcAtllSB 1 0 2O&B, WHltfavyK 

ttfst j-sfrr 6 ttuz. Bs&nuie**-*- &*# 

im. $mmx*wynz£m 

-^zmm^tii . 

[0035] 03t^tJ:5fc, y-A^H^« 

zmsttitmbiztmnrntzz «t 3^n!ft«c3 1 

-3 35rE»f ,1^313^3 1-3 

3 co^tta^fflatcsi'f £ mi t s^m^s^iis 1 

[ 0 0 3 6 ] m3 ( a ) {iftSTU— \**>^^y-A 

7*-jh« («-^n^fsi) <r>mim. m3 (b> <±^- 

O) r-y;P3 0<0JiC 3o«o^SISft*3 1-3 
3£|$$fti:LTEBU:*&££5jrr. *Htfijg)Bi^ 
v^TJi. ±a<0J: oHitlga^A^liai 0 2 frfflv*: 

Ttiitmb. Hsmcmii.zto^xmn&znm.nisz 
& T-rtu 3 o n&zmmmssmt: eb lt h 

[00 37]H3(CJ3V>T. «H23WWIc3 1, 3 2Ji 
3 3«^T»i:$tlTU4. ^»<03SHSI3«i«c*< 

S. ^T»3S3|!ft«c3 3*<7^rA^Ae-H-C 

W^^Bj*3 1. 3 2m*«B$nTV^^-J:0 

<m& i tf<m l . t titmm&imm 3 3 izm 
Kmm±>v3 3 1 *wmmmm&2izx ~> 
xftmthztizzi. 
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~'*-i'ATMZ±',X$&%tlZbtfX'$ S. fli^co 
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h z t izx o , r v-rmmz m.iznmmx*> 
mz£*)mx%h. 
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mz x 1 tiMttfut ttw mm®»<r)m*tf'te v * 1 7 
[0040] (v-^Eir^a) 04ji, *mmmz 

mny-Mzts^xii. %mimiz£&f$mmmtt 
*nwmizmmix-?-tizwxthzbi l zi 

0. #tv— rn&mmftizti^xmfizAi,-?-/, 

[0041 1 05 (a) - (c) ii-e*i-e*u m4iza 
ttiru— \»a, b, ctpt*Khzbr>X'$ hi-ii* 
fornix fjkifz®X'h 1. ;(0i3c. 

Zbi>^mb%&. 

[0042] 06 (a) - (c) li. K«ft£*«JEL& 
H*£te#7V-- f*><i>fi&3*l67-;$£05 (a) 

~ ( c ) £*«eLT^U:0T**>S. *5%ajtzJ:|)7- 

I i : Aft4?HMU 
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ITH i : 2ffiB<SU7)i 
Rm i n A, Gm i nA, BminA: 

RmaxA, GmaxA, BmaxA : 

b-fhb. &I iZbtZ. Rmi nA<Ri<Rmax 
AAoGra i nA<G i <GmaxA*>OBm i n A< 
B i <BmaxA^t I i I TH i iZ 

1, W^kO I UCftjEtSITHifcOfcLT. ~ 
ffl|«ITHt»rtW-4. 

[0048] ov^T, -filHIR I THlZyKV y?W£ 
S703) . -E-LT. #;7X^Oi-t> (Xn. Yn) 



[0043] ru-wmaitzmnzttv-trtm 

0. &r&1^h%b'<ni3mX'tt®tlZb&X'$l. 
&*®*i?ti$r£i>t£$:<D&l l z l l fefcfOO £ 

t & £Eat£ £ i: mmb . 

[0044] ^-^<oEa(ait<o^ii, xmsm->x 
i^fflmm^mmn^T/vi&mix&z. n&b 

i><T&t>tih9m&jft#>$>Zbtz£r>x&&?&Zbti s X- 
Zl. mit:^-?-ti<MSib&XMM&m^X 
&&&*lli-ti>£o£l,Xt>&\,\ Z<r)£o%& 

hioiizmnLixh&w 

[ o o 4 5 ] a*, mmwsb^-tnwm.i.zmmx'* 

®te£Mm®.XVb$Z b Hz J: 0 „ r Wi-?-#/fltf> 

[0046] ( v-^cotaai) »:fc:v-^cottai^rat 
tmmwxzwctyv— fx-YX'bz. 

E££ff-iI!a5lOl#fT-5. 

[ 0 0 4 7 ] £1\ hT^510 3T«»$^i 
B«A^llBl 0 45r^-LTK#1-« (Xf77S 
70 1) . -f-LT. 2fiftJ!iaSrffo (^f77S7 0 
2) . 07 (W<0«Ui^LTie») ^ 

$^Sa«A(^iiiSB*$- 1 , ttiimtOb? 

$r««-f S i #g<0g* 
, G, B#fe^ffl 
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R. G. B^n-Pil^S^ffi 
i:ffi«anS:KaiLT (Xf7TS704) . SSSfiia 
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[0 04 9] 08tC*^vCtt, &*1£ <r>7- 

[00501 ( -?-*m* > m&om o , v-^ji* 
*fss5{aa«»-b ^-t i o 1 1 oaj^*<jEirc*#uf^ 

[0051] *»fl(CJS^Ttt, ru-^iz 

[0052] H9«. v-*mm&jm*Bmti&? 

*>h. 09 (a) fci, 43KNffinfc&ft«'?--4EK£ > 

■c. as (a) twjcts. co«tatcf— r/w&t/w 

-Xr*>KBSL 7"U— \*c7)HMD KH^f I) H«T - * 

rof#LT*5v^T^x^s«-ca^8i^4*\ t# i. 

?m7*-?*Jt*?& (09 (b) ) . 

[00 53] ZVJio£WiglT-?t:&$mz./ma:? 

t ixMBtiwmT-m. nmzmzti&m 

a*® 5r<s»r i ±.xnft i v m t ttmvwm* 
izMmzixi-?-ti£m+iT?*i-'rwm$im 

£MT?x+*klXi>£\\ 
[00 54] 4fc. KWftSrtefflLfr^TTy^-^a 
yfcfcwc, v-#Eao*:tf><DA£Jil&1%#$:EBL 
^io&ii^K ffl^^v-^tJtLTBtflT-^S-a 

#«<t/ag-r*^ait^^^i:*{i, 09 <c> <r> 

w<7)«^, k'OXoZWmT-ftm^ 

T7vy-i,*vizi.'>xms.%)f>iMiX\\ 
tzkut. y-j*Try*-is!a>izti^x\t. mzz 

nmm9 1 zmmix. msfsizm&ZT-ism 

Kch^im&LXii^L, W&15k<rmt:&tZlt 

^<^^7'jt-y 3 yx'btu&sm&isxwM 

?X%<. 3dttm<r)T-9k'thZ.ki>-aS&X'h 



[0055] 

Mzm^x^titf. m&m£$±>in 0 1 1 tm* 

^O^tta^^^S^tnTfigT* 0 , 

ate t nataE "Jig?*. * c k am ? t x t> 

[ 0 0 5 6 ] ±»*>«ttB!Wc*Nvcii. hl> # 
10 2 2 *ffifiH*-f s ^S^A^SiawlftS $• fcffl L 
T 3 7V H * A^3 U:#, 7V~\r# M> # 1 0 2 2 £ 

tt^xzn^yvtxjj-t&xoizixh. mmm 

bhVtf<r>*y< *7k*m&lrbitXi>&^. 

fc. *^»f^A^satc yv-^com^^zmtx^y 

• *ytl£o%*4 v*£&A.&&XbVtfk IXM 

\>\&zki>x$z>. 

too57] *m&>m\i.. ±MLt:mmm^mn 

( 4*ttcpuwu) #mmmzmiztit;Tar?A 
z-vzmx-mimr-tizkizzix^ m&ztii 

tzmm\t*%mt:n&thzk(iz%h. n> 

fix-nmix^i*^— T4 y^^rA(os)^t* 
x m 1 z mm®*) tm#mi $ ti 1 ®$ t # 4 ^ ^ 

[00 58] £1«|ff^4>R»a{$^7D^ 

\ : *3>z*-9iz®mtit:mmm3--v hizmb 

£l>*iti&Zkli.Wo*Xi>*\\ 
[0059] 

&£mmimzt3^x&m*>bit<ot:it)tzm*t>tii 

WfclKtfbWStbZkimifcktch. tfz. 7V- 
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[0060] %t>tz. i>ti>t&,mztit:i$££iaiim 

zwm-fi z t htz#> . rry T->- a yfc «t -> 
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[06 3 R3ft*fflfflH*-f £04 bm*w-*zsm 

[07 j ^mtrmmmiziivz. 

[08] ^5-S«T-^*^07tC^-rfe««CO'7- 

[09] v-tKDii&ijmmty-rmx'&h. 

[01 0] -?-^^>fea$rflH>L^|g^m«ai(CO^T 
iKBWS0Ti>4. 
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